Hyper-Graph Based Database Partitioning for Transactional Workloads

A common approach to scaling transactional databases in practice is horizontal partitioning, which increases system scalability, high availability and self-manageability. Usually it is very challenging to choose or design an optimal partitioning scheme for a given workload and database. In this technical report, we propose a fine-grained hyper-graph based database partitioning system for transactional workloads. The partitioning system takes a database, a workload, a node cluster and partitioning constraints as input and outputs a lookup-table encoding the final database partitioning decision. The database partitioning problem is modeled as a multi-constraints hyper-graph partitioning problem. By deriving a min-cut of the hyper-graph, our system can minimize the total number of distributed transactions in the workload, balance the sizes and workload accesses of the partitions and satisfy all the partition constraints imposed. Our system is highly interactive as it allows users to impose partition constraints, watch visualized partitioning effects, and provide feedback based on human expertise and indirect domain knowledge for generating better partitioning schemes.